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Abstract 

This article describes the use of a schematic model for developing and 
distributing online surveys. Two empirical studies that developed and 
implemented online surveys to collect data to measure satisfaction in various 
aspects of human resource development and adult education exemplify the use of 
the model to conduct online survey research. The steps of the model used for 
online survey development and implementation, descriptions of the two case 
studies, and recommendations for the use of online surveys are presented. 

The gathering of feedback is an important part of educational program development and 
workforce development in general. It is critical that human resource development (FIRD) 
practitioners and adult educators are able to gauge the success of their efforts from the 
viewpoints of a wide variety of stakeholders. One of the best ways to gather learner, employee, 
and consumer feedback for research, evaluation, and continuous improvement is through the use 
of surveys. Telephone surveys or hard-copy surveys, mailed or manually distributed to potential 
survey respondents, have traditionally been used to gather survey data. In the age of the Internet, 
however, online surveys can be an efficient way to gather data as well (Strachota, Schmidt, & 
Concei^ao, 2005). 

Despite the myriad of opportunities offered by the Internet, survey researchers have not 
been leaders in the use of Internet-based surveys. In fact, Dillman and Bowker (2001) note: 
the use of web surveys seems to have caught the survey methodology community 
somewhat by surprise. Leadership for the development of web survey procedures has 
come in large part from computer programmers, many of whom have little or no training 
in survey methodology. ..rather than being at the forefront of this latest innovation in the 
conduct of social surveys, survey methodologists are playing catch-up as they learn to 
master these new survey development tools, (p. 1) 
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Using the Internet as a data-gathering tool adds extra layers of complexity to the process 
of survey development. Gunn (2002) suggests that “unlike other types of surveys, web page 
design skills and computer programming expertise play a significant role in the design of web- 
based surveys” (p. 1). These challenges have been made somewhat easier by the development of 
websites specifically geared to help researchers develop and distribute online surveys. Websites 
such as SurveyShare.com, SurveyMonkey.com, and Zoomerang.com allow researchers to set up 
online surveys in a matter of minutes. Many of these sites offer distribution, tabulation, and 
reporting features as well. 


Statement of the Problem 

Gathering data online for research projects has become easier provided that the available 
tools are used properly. However, researchers must be aware of additional considerations 
associated with the use of online surveys that should be addressed as part of survey development 
and distribution processes (Strachota et al., 2005). This article suggests the use of a schematic 
model for developing and distributing online surveys. This study addresses the following 
research questions: 

a) How can online surveys effectively be used to gather data for evaluation and research 
within the fields of HRD and adult education (AE)? 

b) What are the benefits and limitations associated with the use of online surveys related to 
HRD and AE evaluation and research? 

c) What factors regarding survey development and distribution must be considered when 
surveying in an online environment? 

This article will contribute to the body of knowledge on online survey development, 
which will help practitioners and researchers in the fields of HRD and AE. Two empirical studies 
that developed and implemented online surveys to collect data to measure satisfaction in various 
aspects of HRD and AE exemplify the process used to conduct online survey research. A 
description of each study, the steps used in online survey development and implementation, and 
recommendations for the future use of online surveys are presented. 

Model Foundation 

The schematic model presented in this work, the Model for Online Survey Development 
is based on the work of Dillman and Bowker (2001), Dilhnan, Tortora, and Bowker (1998), Rea 
and Parker (1997), Wallen and Fraenkel (2001), and their contributions to survey development. 
Wallen and FraenkeTs (2001) process includes the definition of the problem, identification of the 
target population, determination of mode of data collection, preparation of the instrument, 
collection of data, and analysis. Also important in their process of survey development is the 
identification of large categories of issues (or constructs), which can be used to suggest more 
specific issues within each category, which can then be used for question development. Rae and 
Parker’s (1997) stages of the survey research such as the importance of the literature review, 
construct development, and validity checks were incorporated into the model. Works by Dilhnan 
et al. (1998) and Dilhnan and Bowker (2001) form the theoretical bases for the online 
components of the online survey development and distribution model. 
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Online Survey Development Model 

This article describes an effective model for online survey development (see Figure 1). 
Two case examples will illustrate the process for the online survey development model. The first 
case is in the context of HRD, specifically corporate job training, and the second case is an AE 
program offering online courses for adult learners in a technical college. 



Cost Efficiency 
Time Efficiency 
Methodological Efficiency 


Figure 1. Model for Online Survey Development 

Steps One and Two: Review of Literature and Research Question Development 

The first step in the research process is to determine the significant aspects of a research 
problem that call for investigation. A review of the literature can assist in clarifying issues and 
framing problems more clearly. Research questions based on the literature are then developed 
(step two) and serve as the foundation of inquiry for the survey instrument. Wallen and Fraenkel 
(2001) suggest that good research questions should be realistic and actually researchable by the 
researcher; they should be clear, significant, ethical, and suggest a relationship of some sort. 
Hypotheses may also be tested using this model. Hypotheses may be stated as such, or they may 
be stated in question form. 
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Step Three: Developing and Defining Constructs 

A construct is a concept measured by multiple items in a survey instrument. Following 
the development of the research question, the researcher compiles a list of key issues or 
constructs that comprise the research questions. Each construct must be defined and clearly 
differentiated from other constructs. 

Creating an original survey instrument requires a theoretical foundation in the area of 
study. The literature review noted in step one serves as the foundation for construct development. 
The literature review will assist the researcher in understanding and appreciating the depth of 
research in a given field, such as HRD or AE, as well as in realizing those areas that continue to 
require further research. 

Rather than rely on personal opinion, and in order to ensure content validity, which will 
be addressed later in this process, the researcher should organize a team of experts to assist with 
construct development (Trochim, 2002). Rea and Parker (1997) recommend that such a team 
“represents both technological expertise and substantive knowledge of the political, 
socioeconomic, and cultural environment associated with the project” (p. 28). Working with the 
above resources, the researcher formulates constructs based on past research and on the team’s 
feedback, with the constructs adding to the field of study by supporting existing theory and 
practice, and also by creating new directions for theory and practice. 

Step Four: Survey Question Development 

Based on the constructs developed and defined in step three, a set of survey questions are 
developed. Ideally each construct should have a minimum of three to five questions that measure 
that construct. Developing more questions for a particular construct is preferred over not having 
enough questions, as questions are often eliminated as part of the factor analysis statistical 
process (Gable & Wolf, 1993; Strachota, 2003). Rea and Parker (1997) note the following tips 
for survey question development: 

1. Keep in mind the population to be surveyed when choosing words, colloquialisms, and 
jargon for survey questions. 

2. Make every effort to avoid ambiguity or vagueness in writing survey questions. 

3. Avoid multipurpose questions, or questions that request one response for two or more 
issues. 

4. When developing explanatory statements or survey directions, take care to avoid 
manipulative or biasing information. 

5. Be mindful of the use of bold, italicized, capitalized, or underlined words, which may 
place inappropriate emphasis on those words, thereby biasing responses. The use of 
emotional words and phrases may do the same. 

During the investigative process, existing survey instruments may be found, and used 
when appropriate. Many survey instruments, however, may not be adequately grounded in theory 
and practice or not designed to adequately answer intended research questions. The validity and 
reliability of the survey instrument is also important, or the data collected is of little value. For 
these reasons the researcher may find that creating an original survey instrument is necessary. 
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Case Examples of Survey Question Development (Step Four) 

In Case 1, a study by Schmidt (2004a) employed the use of an existing survey instrument. 
The job satisfaction aspects of this survey originate, in part, from Paul Spector’s (1997) 36-item, 
nine-factor Job Satisfaction Survey. Spector’s nine subscales (or constructs) measure satisfaction 
with pay, promotion, supervision, fringe benefits, contingent rewards (performance-based 
rewards), operating procedures (required rules and procedures), coworkers, nature of work, and 
communication. There are four questions for each subscale. Spector’s survey was used because it 
was a well-tested and readily available existing instrument that measured the topic (job 
satisfaction) of interest to the researcher. 

In Case 1, the Job Training facets of Schmidt’s (2004a) survey Job Training and Job 
Satisfaction Survey measured employee satisfaction with on-the-job training. The three subscales 
(or constructs) measured organizational support for training, employee feelings about training, 
and employee satisfaction with training. Specific questions for each of these constructs were 
developed by Schmidt (2004a) based on learning organization assessments (Wick & Leon, 

1993). Four questions were developed for each subscale. Schmidt’s (2004a) survey also 
contained five questions regarding training methodology, training content, and time spent in 
training. Seven demographic questions were included as well. 

The original survey instrument developed by Strachota (2003) for Case 2 included 33 
questions. Nine questions measured the construct of learner-content interaction. Learner- 
instructor interaction, learner-learner interaction, and general satisfaction were each measured by 
eight items, respectively. A four-point Likert scale was used. Selected questions were obtained 
from the Cassidy and Eachus (2000) computer self-efficacy instrument to measure learner- 
technology interaction. In addition to the constructs that were measured, the survey instrument 
included a rank-ordering question. Students had to rank-order the following: learner-content, 
learner-instructor, and learner-learner interaction as to which they perceived as 1 ( the most 
important), 2 ( moderately important), and 3 (least important) to a satisfying online experience. 
Following the rank ordering was an open-ended question asking students why they ranked a 
particular construct as most important. Four additional open-ended qualitative questions were 
included in the instrument specific to online learning. Eleven demographic questions were also 
included. 

Steps Five Through Seven: Validity Check, Pilot Study, and Factor Analysis 

The validity of an instrument is concerned with the question, “Does the instrument 
measure what it is supposed to measure” (Gable & Wolf, 1993, p. 95). Wallen and Fraenkel 
(2001) state that “validity is the most important idea to consider when preparing or selecting an 
instrument for use” (p. 88). Validity includes the appropriateness, meaningfulness, and 
usefulness of specific inferences made by the researcher based on data that are collected (Wallen 
& Fraenkel, 2001). The degree to which those inferences are correct or appropriate is dependent 
on the validity of the research instrument. An effective instrument must have both content 
validity and construct validity. Content validity, step 5 in Figure 1, or what is often referred to as 
face validity, is judgmental in nature and is measured by having experts in the field examine the 
instrument to ensure that all test items appear to address the purpose of the instrument and the 
variables that are to be measured. As noted in step 2 (above) the same team of experts that 
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advised the researcher on construct development should be called upon to determine if any of the 
questions are unclear, to modify items, to eliminate items that do not appear to be a measure of a 
given construct, and to create additional items if necessary. Once the instrument has established 
content validity a pilot study of the instrument, step 6, should always be conducted. Factorial 
analysis of data from the pilot test will assist in determining construct validity. 

The purpose of construct validity is to detennine if the constructs being measured are a 
valid conceptualization of the phenomena being tested. Construct validity is determined from the 
data analysis of the instrument. According to Gable and Wolf (1993), “construct validity focuses 
directly on response-data variation among items to ascertain evidence that the proposed content 
categories actually reflect constructs” (p. 101). Construct validity is empirical in nature and is a 
critical step in the process of establishing instrument validity. Data from the pilot study are 
analyzed through the use of factor analysis, step 7, to determine if indeed given items load on the 
intended construct. Items that do not load on the intended construct are eliminated, as they are 
not an adequate measure of that construct. Therefore, administration of a pilot study is critical to 
the validity of the final instrument. According to Gable and Wolf (1993), “factor analysis is used 
to examine the relationship between judgmentally developed content categories and the 
empirically derived constructs” (p. 106). 

Instrument reliability is concerned with the question: “Does the instrument provide us 
with an accurate assessment of the affective characteristic?” (Gable & Wolf, 1993, p. 201). 
Reliability refers to the consistency of scores or answers from one administration of an 
instrument to another and from one set of items to another (Wallen & Fraenkel, 2001). 
Conducting a pilot test of the survey instrument will also detennine its level of reliability or 
Cronbach’s alpha, an internal consistency or reliability coefficient for instruments requiring only 
one test administration (Wallen & Fraenkel, 2001). Internal consistency reflects the degree of 
homogeneity between test items such that items are a measure of the same construct. Cronbach’s 
alpha scores range from zero through one, with a coefficient closer to one indicating higher 
reliability. Reliability coefficients should be at least .70 or higher to be considered reliable for 
affective instruments (Gable & Wolf, 1993; Wallen & Fraenkel, 2001). A survey instrument 
must demonstrate both validity and reliability to be considered an effective measurement tool. 

Case Examples of Validity Check, Pilot Study, and Factor Analysis (Steps Five-Seven) 

In order to ensure validity of both the Case 1 (Job Training and Job Satisfaction) and 
Case 2 (the Online Satisfaction) survey instruments, the following steps were taken: experts 
within the field reviewed instrument items for clarity, precision, and revision to ensure content 
validity. Following content validity, a pilot study was conducted by both researchers to test their 
instruments for construct validity. Using the data obtained through the pilot study, a factor 
analysis was performed to evaluate the factor loading of the constructs within each instrument. 
Although pilot testing an instrument is critical to the process of developing a valid instrument, 
this step unfortunately is often eliminated. However, it is the step that ensures that the instrument 
truly is an accurate measure of what it is intended to measure. Items that do not factor load under 
the intended construct should be eliminated as they are not an effective measure of that construct. 

A pilot study (N= 118) was conducted for the Job Training and Job Satisfaction Survey. 
Inappropriate questions were deleted based on information collected in the pilot, and when 
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combined with questions regarding job training, the factor structure of Spector’s (1997) original 
Job Satisfaction survey differed slightly from that of the original. A six-factor solution was 
obtained rather than the eight constructed by Spector (1997). The original subscales of 
satisfaction with pay, promotion, and contingent rewards were combined to form a single 
subscale that was called “Satisfaction with Opportunities and Rewards.” Another subscale, 
“Satisfaction with Communication,” was removed entirely. The remainder of Spector’s original 
subscales engendered a relatively clean factor structure with limited overlap between the scales. 
Reliability testing of the pilot study indicated a Cronbach’s alpha of .89 for the revised job 
satisfaction scale and a Cronbach’s alpha of .83 for the job training scale. 

The same pilot process was carried out for the Online Satisfaction Survey (N= 249). 
Through factor analysis the final instrument was reduced to 27 items as 6 items had low factor 
loading that overlapped across all constructs, indicating that they were not a good measure of 
that specific construct. The final instrument included seven items that were a measure of learner- 
content interaction, six items that were a measure of learner-instructor interaction, eight items 
that were a measure of learner-learner interaction, and six items that were a measure of general 
satisfaction. Removal of the six items resulted in moderate to high factor loading with the 
appropriate items loading within the specified construct. Therefore, questions within each 
construct were considered to have good internal validity. Reliability testing of the pilot study 
indicated a Cronbach’s alpha of .90 for the constructs of both learner-content interaction and 
general satisfaction. The constructs of learner-instructor interaction and learner-learner 
interaction resulted in a Cronbach’s alpha of .89. Items specific to learner-technology were not 
pilot tested as they had been piloted tested by Cassidy and Eachus (2000). They showed a 
Cronbach’s alpha of .97 for the single construct of computer self-efficacy, which was equated as 
being synonymous with learner-technology interaction for the purposes of this study. 

Step Eight: Adapting the Survey for Internet Use 

The development of an online survey instrument begins with the design of the survey 
itself. While this process may not have changed with the advent of the web-based survey, factors 
in addition to those associated with traditional survey development should be considered when a 
survey is put online. Dillman et al. (1998) recommend that “questions must be presented in a 
way that they can be understood and answered accurately by all recipients. And, the likelihood of 
responding should not be affected by peoples’ lack of computer skills” (p. 3). To address these 
issues, Dillman et al. (1998) advise online survey developers to consider the following: 

1. Survey design should be respondent-friendly and take into account the inability of some 
respondents to receive and respond to web questionnaires with advanced programming 
features. 

2. Survey design should consider both the logic of how computers operate and how people 
expect questionnaires to operate. 

3. As many survey populations do not have access to the Internet, a survey design should 
allow for survey use in other modes of data collection (i.e., paper questionnaires, 
interviews, or telephone surveys). It is also important to remember, from a reliability 
standpoint, that those other modes of data collection be administered using as many of 
the same guidelines as the online survey as possible (i.e., the time frames in which the 
surveys are administered and the location at which surveys will be completed should be 
similar for both online and non-online respondents). 
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Several additional factors should be considered when developing a survey for Internet 
use. Confidentiality must be protected in the data gathering process as well as in the data analysis 
process. 

When detennining the best way to distribute a survey, issues of cost, time, and 
methodological efficiency should be considered. The researcher must also consider issues 
associated with the distribution of web-based surveys to potential respondents. According to 
Timmerman (2002), participants are usually asked to participate in web-based surveys via e-mail 
messages, which often include a URL (Uniform Resource Locator) link to the survey website 
itself. 

Case Examples of Adapting the Survey for Internet Use (Step Eight) 

Case 1 (Schmidt, 2004a) and Case 2 (Strachota, 2003) followed the recommendations of 
Dillman et al. (1998) when developing their online surveys. To answer Case 1 research 
questions, a convenience sample of 552 customer and technical service employees in nine major 
organizations in the United States and Canada were asked to complete the Job Training and Job 
Satisfaction Survey. Organizations chosen for this study employ customer service and technical 
service representatives who provide service and support to either end (retail) customers or 
wholesale customers who both purchase and/or service the products or services provided by 
these organizations. All potential respondents used computers in their jobs, so the technical skills 
of the survey audience were considered to be at an acceptable level. Examination of the 
technology available to potential survey respondents revealed that several organizations that 
agreed to participate in the study did not provide Internet access for their employees. An 
alternate method of distribution was adapted in the form of a hard copy survey distributed 
manually to employees in those positions. 

To answer Case 2 research questions, a convenience sample of 1,593 online students at a 
technical college in the Midwest area of the United States were given the Online Satisfaction 
Survey. Students were enrolled in various online courses as part of the program requirements for 
an associate degree in preparation for employment. The Online Satisfaction Survey was accessed 
through a web link in the college’s Learning Management System (LMS) that hosts all online 
courses. All survey participants had access to the Internet either from home, work, or the college 
computer labs. All data were collected via an electronic format. 

Step Nine: Survey Distribution 

Surveys can be distributed in a variety of ways. They can be sent via traditional mail, 
conducted over the telephone, distributed in person to potential respondents, and sent via e-mail. 
As noted above, it is important to consider issues of cost, time, and methodological efficiency 
when detennining the best method of survey distribution. 

Case Examples of Survey Distribution 

For Case 1, the Job Training and Job Satisfaction Survey was posted on an Internet 
survey website. Potential respondents with Internet access were e-mailed the survey link and 
asked to complete the survey online. Those without Internet access were given paper copies of 
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the survey. The researcher employed a procedure (Schaefer & Dillman, 1998) for sending e-mail 
invitations to potential survey respondents, a procedure regularly used by the Social and 
Economic Sciences Research Center. It consists of four contacts via e-mail: 

a) A pre-message, requesting potential respondents to watch for the survey link. 

b) A message that contains the link, along with survey instructions. 

c) An e-mail thank-you note or a reminder, which also includes the link. 

d) A final reminder, also with a link to the survey. 

The researcher conducted this procedure over a 2-week time period. A total of 301 
customer/technical service employees completed the Job Training and Job Satisfaction survey, 
for an overall response rate of 55%. Broken down by methodology, the researcher invited 263 
employees to complete hard-copy surveys; 177 employees did so, for a response rate of 67%. 

The researcher sent 289 e-mail invitations to potential online respondents, and 124 invitees 
completed the survey online, for a response rate of 43% (Schmidt, 2004b). 

For Case 2, the Online Satisfaction Survey was posted as a web link in the college’s 
LMS, which houses all of the online courses. Each time a student would log into their course 
they would see the survey link. Besides placing the link in the LMS, the researcher also 
incorporated four e-mail contacts to students. The first contact included a pre-message requesting 
potential respondents to watch for the survey link in the LMS. One week after the link was 
posted in the LMS, an e-mail reminder message including the survey link and instructions was 
sent to all online students. The third contact included an e-mail thank-you note or a reminder, 
which also included the link. Following this e-mail message a final reminder was sent with the 
survey link. The researcher conducted this procedure over a 4-week period of time. A total of 
849 students completed the Online Satisfaction survey, for an overall response rate of 53.3%. 

These two empirical cases illustrate the benefits gained by following the processes 
identified in the Model for Online Survey Development. For both case studies, the online survey 
response rate exceeded the averages found by Sheehan (2001), who examined 31 online survey 
response rates from 1986-2000 and found the average response rate for those surveys to be 
36.83%. 


Recommendations for the Use of Online Surveys 

The process of using online surveys for collecting data to measure satisfaction in job 
training and distance learning in two empirical studies allowed researchers to identify some 
advantages, limitations, and suggestions for the future use of online surveys. 

Advantages of Online Surveys 

When compared with traditional mail or telephone surveys, Schonlau, Fricker, and Eliott 
(2001) note that Internet surveys may be preferable to mail or telephone surveys when the survey 
contains questions of a particularly sensitive nature. The use of Internet-based surveys may allow 
the program developer to delve deeper into evaluation-related topics and result in more honest 
feedback than would traditional survey methodologies. 
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Advantages of online survey distribution also include increased time efficiency, 
decreased data entry error, increased item response rate, and decreased cost. Due to available 
technology, administration of online surveys has resulted in increased time efficiency in both 
distributing and receiving of survey results. The majority of colleges, universities, and businesses 
today that deliver distance learning courses and programs have invested in an LMS. The LMS is 
designed to allow for the posting of an announcement with a direct web link to an online survey 
instrument. The LMS offers efficiency for students who are enrolled in a course because they can 
access the survey instrument and complete it at a convenient time. The LMS also contains a 
database of e-mail addresses for all students. When the LMS is not available, the process 
becomes far more time-consuming as e-mail addresses must be obtained before a mass mailing 
can be conducted. Both of these methods, however, still remain far more time efficient than 
telephone or mail surveys. 

Decreased data entry error is an important factor when conducting survey research. 
Traditional telephone and mail surveys require manual data entry. Electronic survey distribution 
does not require manual entry as data are collected through a given software program and then 
downloaded into a statistical program such as SPSS for detailed analysis. This feature saves time, 
is more accurate, and is especially useful when collecting data with a large sample size. 

Increased item response rate can also be controlled through electronic surveys, as most of 
the software programs can be set so that all items must be completed before submission. 
Traditional mail surveys often are returned with data being absent in given fields. Further, 
qualitative item response rate can be increased due to the legibility of electronic responses versus 
occasional illegible mail responses. Schaefer and Dillman (1998) found that electronic surveys 
resulted in a quicker return rate, a slightly lower item non-response rate, and more complete 
answers to open-ended questions. 

Another advantage of electronic survey distribution is decreased cost compared to 
telephone and mail distribution. A major concern of survey research is the potential for high non¬ 
response error. In order to minimize non-response error, multiple contacts are made to have 
participants complete a given survey. Schaefer and Dillman (1998) support the use of multiple 
contacts as a method for survey dissemination as it increases the response rate. Informing 
participants that a survey will be forthcoming through an announcement; presenting the actual 
survey link, a thank-you, or reminder e-mail with the survey link; and sending a final e-mail 
reminder with the survey link can all be entered in an LMS and/or e-mailed at no cost. 

Contacting participants multiple times through mail or telephone distribution, however, is far 
more costly and time consuming. Therefore, electronic survey distribution is a far more efficient 
and effective method for data collection and evaluation of educational and training programs. 

Limitations of Online Surveys 

The online survey must have user-friendly navigation. Certain computer skills such as 
selecting radio buttons, using drop-down menus, and scrolling are commonly required skills. 
Correctly reading the directions and paying attention to detail is also important. Dillman and 
Bowker (2001) identify the need to provide specific directions for each required computer action 
as well as any other necessary instruction to avoid frustration and termination of the survey 
before completion. 
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A potential limitation of e-mail survey research includes the difficulty in sorting out 
responders versus non-responders, especially with a large sample size. Annoyance may 
occasionally be perceived in those receiving multiple e-mails requesting a response. E-mail, like 
mail delivery, may be perceived as junk mail and deleted without responding. In order to achieve 
higher compliance among potential responders, some incentive should be established as part of 
the process. Incentives may include access to online books or reports of interest to potential 
respondents or small thank-you gifts (Strachota et ah, 2005). 

Conducting online surveys versus telephone or mail surveys is a good fit for those who 
use a computer as a common mode of communication as well as for those who enroll in online 
learning for career and job advancement. Although conducting online surveys is far more 
efficient and cost effective, it is important to recognize the audience being surveyed. Although 
we live in a technological age, we are still faced with the issue of Internet access. While 54.6% 
of the U.S. population has Internet access in their homes, and an additional 8.4% who do not 
have Internet access in their homes have access elsewhere, 37% does not have Internet access 
(Cooper & Gallagher, 2004). 

Some topics, for reasons including those associated with sensitivity of subject and 
confidentiality of data collected, may best be researched via different means. Further, some 
research studies may be designed in such a manner that the researcher feels than mail or phone 
remain the better mode of delivery. Timmerman (2002) suggests that “before assuming web- 
based surveys are the most advantageous method for surveying a target population, it should be 
determined if those advantages exist within that population” (p. 5). 

Suggestions for Future Use of Online Surveys 

Survey distribution is a critical variable in the process of data collection. Online survey 
distribution typically involves the use of an e-mail message sent to potential respondents, which 
includes a link to the survey’s URL. Following are seven recommendations for effective and 
efficient survey distribution using e-mail. These recommendations are based upon the authors’ 
experiences with online surveys as well as conclusions made by other researchers: 

1. E-mail address and list management is important. Be sure all potential respondents’ e- 
mail addresses are complete and spelled correctly. This will reduce the number of 
undeliverable messages to be corrected and sent again. 

2. Develop a method to track whether or not all potential respondents received the e-mailed 
survey invitation. Send test messages to a variety of people with different e-mail 
addresses (i.e., business or organizational addresses and personal addresses) before 
sending to potential respondents. 

3. Creative and informative information in the e-mail subject line will prevent potential 
respondents from thinking that the researcher’s e-mail message is spam or junk mail and 
simply deleting it. 

4. E-mail filters used by many businesses and individuals may prevent e-mail messages 
from getting to potential respondents. Online survey websites such as those mentioned 
earlier have the advantage of allowing the researcher to send e-mail survey invitations to 
many people at once. However, e-mail filters may recognize the return address (which is 
often the survey website) as spam. This may cause the survey invitation to be diverted 
from potential respondents. Learn about any potential issues with filters ahead of time so 
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e-mail invitations can be sent in smaller quantities at one time (some filters filter out 
blanket e-mails sent to large groups) or sent to alternative e-mail addresses. 

5. Ideally, survey respondents will access a web-based survey by clicking on a URL link in 
the body of the e-mail. Be sure all li nk s work before sending them to potential 
respondents. Testing e-mail messages can be effective in ensuring a link works. 

6. Have a backup to the URL link. Instruct survey respondents that if the link does not 
work, they can copy the link and paste it into the web address line. Also, provide 
instructions for accessing the survey by providing the website address to the online 
survey tool, along with instructions for respondents who go directly to the website 
(Schmidt, Strachota, & Concei^ao, 2005). 

7. E-mail messages and survey links should be thoroughly tested before any survey 
invitations are sent to potential respondents. Send test messages to as many different 
types of e-mail addresses as possible and to potential respondents with a variety of 
different browsers and Internet systems. Along with those test messages, send li nk s to 
test surveys on the survey website. This, again, will ensure that as many respondents as 
possible receive the message, access the online survey using the link provided, and 
complete the survey. 


Conclusion 

This article presents a Model for Online Survey Development and Implementation for 
conducting online survey research. Critical to survey development is the need to conduct an 
extensive literature review to serve as the foundation in the development of research questions 
and for construct development. Survey research is an effective research methodology that can 
contribute to the fields of HRD and AE by both supporting existing theory and practice and by 
creating new directions for both fields. All survey instruments should be pilot tested to ensure the 
highest degree of validity and reliability. Both the Schmidt (2004a) and the Strachota (2003) 
survey instruments were showcased to demonstrate the importance of pilot testing and factor 
analysis. 

Due to the technological age in which we live, online survey research may become the 
preferred delivery method over mail and telephone survey distribution. Before deciding to 
conduct survey research online one must be aware of the audience to be surveyed, their computer 
skill level, and their access to a computer and Internet. 

Online survey research is far more appealing to the researcher who may have time or 
financial restrictions. Knowledge and application of online survey software and technology is 
required for this methodology. The efficiency and cost effectiveness of conducting online survey 
research, however, far outweighs the time and efforts required to learn and effectively utilize a 
specific software program application. 
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